PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalization

Deep neural networks (DNNs) are vulnerable to adversarial attacks. It is found empirically that adversarially robust generalization is crucial in establishing defense algorithms against adversarial attacks. Therefore, it is interesting to study the theoretical guarantee of robust generalization. This paper focuses on norm-based complexity, based on a PAC-Bayes approach (Neyshabur et al., 2017). The main challenge lies in extending the key ingredient, which is a weight perturbation bound in standard settings, to the robust settings. Existing attempts heavily rely on additional strong assumptions, leading to loose bounds. In this paper, we address this issue and provide a spectrally-normalized robust generalization bound for DNNs. Compared to existing bounds, our bound offers two significant advantages: Firstly, it does not depend on additional assumptions. Secondly, it is considerably tighter, aligning with the bounds of standard generalization. Therefore, our result provides a different perspective on understanding robust generalization: The mismatch terms between standard and robust generalization bounds shown in previous studies do not contribute to the poor robust generalization. Instead, these disparities solely due to mathematical issues. Finally, we extend the main result to adversarial robustness against general non-$\ell_p$ attacks and other neural network architectures.Comment: NeurIPS 202

Adversarial Rademacher Complexity of Deep Neural Networks

Deep neural networks are vulnerable to adversarial attacks. Ideally, a robust model shall perform well on both the perturbed training data and the unseen perturbed test data. It is found empirically that fitting perturbed training data is not hard, but generalizing to perturbed test data is quite difficult. To better understand adversarial generalization, it is of great interest to study the adversarial Rademacher complexity (ARC) of deep neural networks. However, how to bound ARC in multi-layers cases is largely unclear due to the difficulty of analyzing adversarial loss in the definition of ARC. There have been two types of attempts of ARC. One is to provide the upper bound of ARC in linear and one-hidden layer cases. However, these approaches seem hard to extend to multi-layer cases. Another is to modify the adversarial loss and provide upper bounds of Rademacher complexity on such surrogate loss in multi-layer cases. However, such variants of Rademacher complexity are not guaranteed to be bounds for meaningful robust generalization gaps (RGG). In this paper, we provide a solution to this unsolved problem. Specifically, we provide the first bound of adversarial Rademacher complexity of deep neural networks. Our approach is based on covering numbers. We provide a method to handle the robustify function classes of DNNs such that we can calculate the covering numbers. Finally, we provide experiments to study the empirical implication of our bounds and provide an analysis of poor adversarial generalization

High temperature flow behavior and deformation mechanism of Nb-22.5Cr-2.5Mo alloy

The high temperature compression is performed by isothermal compression tests using Gleeble 3500 thermo-mechanical simulator at a constant strain rate for the hot-pressed Nb-22.5Cr-2.5Mo alloy. The deformation temperature ranges from 800 to 1200 °C and the strain rate ranges from 0.001 to 0.1 s−1. The high temperature flow behavior and deformation mechanism of the alloy are investigated based on the compression test data and TEM observation of the deformed microstructure. The results show that the ductile–brittle transition temperature of the alloy is about 800 °C, and the flow stress is sensitive to the deformation temperature and strain rate, in which the plastic deformation ability increases obviously with the increase of deformation temperature and the decrease of strain rate. Besides, the hyperbolic sine Arrhenius model with stress exponent of 3.131 and apparent activation energy of 402 kJ/mol is established for the peak flow stress. The value of apparent activation energy is higher than that of Nb metal (246 kJ/mol), which implies the existence of Laves phase NbCr2 makes the deformation of the alloy more difficult. Moreover, it is found that the deformation mechanisms in the Nbss matrix are mainly DRV controlled by the glide and climb of the dislocations and DRX formed by bulging nucleation mechanism, while the deformation mechanisms in the Laves phase NbCr2 particles are mainly twinning and synchroshear of the Shockley partial dislocation

Intrusion detection based on k-coverage in mobile sensor networks with empowered intruders

Intrusion detection is one of the important applications of Wireless Sensor Networks (WSNs). Prior research indicated that the barrier coverage method combined with Mobile Sensor Networks (MSNs) can enhance the effectiveness of intrusion detection by mitigating coverage holes commonly appeared in stationary WSNs. However, the trajectories of moving sensors and moving intruders have not been investigated thoroughly, where the impact between two adjacent moving sensors and between a moving sensor and a moving intruder are still underdetermined. In order to address these open problems, in this paper, we firstly discuss the virtual potential field between sensors as well as between sensors and intruders. We then propose to formulate the mobility pattern of sensor node using elastic collision model and that of intruder using point charge model. The point charge model describes an hitherto-unexplored mobility pattern of empowered-intruders, which are capable of acting upon the virtual repulsive forces from sensors in order to hide them away from being detected. With the aid of the two models developed, analytical expressions and simulation results demonstrate that our proposed design achieves a higher k -barrier coverage probability in intrusion detection when compared to that of the conventional designs. It is also worth mentioning that these improvements are achieved with shorter average displacement distance and under the much more challenging MSNs settings.</p

Efficacy of virtual reality training on motor performance, activity of daily living, and quality of life in patients with Parkinson's disease: an umbrella review comprising meta-analyses of randomized controlled trials

Abstract Objective There are several meta-analyses of randomized controlled trials (RCTs) demonstrating the benefits of virtual reality (VR) training as an intervention for motor performance, activity of daily living (ADL) and quality of life (QoL) outcomes in patients with Parkinson's disease (PD). However, the aggregate evidence collected to date has not been thoroughly evaluated for strength, quality, and reproducibility. An umbrella review from published meta-analyses of RCTs was conducted to evaluate the strength and quality of existing evidence regarding the efficacy of VR training in improving the motor performance, ADL and QoL outcomes of patients with PD. Methods PubMed, PsychInfo, Web of Science, and Scopus were searched to identify relevant meta-analysis of RCTs examining the effects of VR training on motor performance and quality of life outcomes in PD patients. We recalculated the effect sizes (Hedges’g) for VR training using DerSimonian and Laird (DL) random effects models. We further assessed between-study heterogeneity, prediction interval (PI), publication bias, small-size studies, and whether the results of the observed positive studies were better than would be expected by chance. Based on these calculations, the quality of evidence for each outcome was assessed by using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) criteria. Results Four meta-analysis with eight outcomes included in the umbrella review was recalculated effect size. Pooled results found VR training can large improve the basic balance ability, moderate improve the overall balance capacity and moderate improve the stride length in PD patients. For ADL and QoL, the effect sizes were pooled that suggested VR training can moderate improve ADL and QoL for PD patients. However, no statistically clear evidence was found in walking speed, motor function and gait function during VR training. The analyzed meta-analyses showed low-to-moderate methodological quality (AMSTAR2) as well as presented evidence of moderate-to-very low quality (GRADE). Tow adverse reactions were reported in the included meta-analyses. Conclusions In this umbrella review, a beneficial correlation between VR and balance ability, stride length, ADL and QoL in PD patients was discovered, especially for the very positive effect of VR on balance because of two of the eight outcomes related to balance ability showed large effect size. The observations were accompanied by moderate- to very low-quality rating evidence, supporting VR training as a practical approach to rehabilitation